AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.352021	4.18
	2_DecisionTree	Decision Tree	logloss	0.220858	43.2
	3_Default_Xgboost	Xgboost	logloss	0.178619	21.47
	4_Default_NeuralNetwork	Neural Network	logloss	0.203019	21.62
	5_Default_RandomForest	Random Forest	logloss	0.202325	20.53
the best	Ensemble	Ensemble	logloss	0.177428	4.32

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.7 seconds

Metric details

	score	threshold
logloss	0.352021	nan
auc	0.5	nan
f1	0.20249	0.10139
accuracy	0.112651	0.10139
precision	0.112651	0.10139
recall	1	0.10139
mcc	0	0.10139

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.352021	nan
auc	0.5	nan
f1	0.20249	0.10139
accuracy	0.112651	0.10139
precision	0.112651	0.10139
recall	1	0.10139
mcc	0	0.10139

Confusion matrix (at threshold=0.10139)

	Predicted as no	Predicted as yes
Labeled as no	0	6853
Labeled as yes	0	870

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

41.0 seconds

Metric details

	score	threshold
logloss	0.220858	nan
auc	0.886197	nan
f1	0.578797	0.451358
accuracy	0.907031	0.49107
precision	0.592233	0.49107
recall	1	0.011272
mcc	0.525154	0.451358

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.220858	nan
auc	0.886197	nan
f1	0.576151	0.49107
accuracy	0.907031	0.49107
precision	0.592233	0.49107
recall	0.56092	0.49107
mcc	0.524228	0.49107

Confusion matrix (at threshold=0.49107)

	Predicted as no	Predicted as yes
Labeled as no	6517	336
Labeled as yes	382	488

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

19.1 seconds

Metric details

	score	threshold
logloss	0.178619	nan
auc	0.945411	nan
f1	0.646284	0.292691
accuracy	0.91726	0.413073
precision	0.667797	0.553569
recall	1	0.000296014
mcc	0.606032	0.27996

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.178619	nan
auc	0.945411	nan
f1	0.642417	0.413073
accuracy	0.91726	0.413073
precision	0.625954	0.413073
recall	0.65977	0.413073
mcc	0.595935	0.413073

Confusion matrix (at threshold=0.413073)

	Predicted as no	Predicted as yes
Labeled as no	6510	343
Labeled as yes	296	574

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

19.7 seconds

Metric details

	score	threshold
logloss	0.203019	nan
auc	0.935024	nan
f1	0.613613	0.360931
accuracy	0.908585	0.516128
precision	0.692308	0.556366
recall	1	9.68502e-21
mcc	0.568529	0.218837

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.203019	nan
auc	0.935024	nan
f1	0.474702	0.516128
accuracy	0.908585	0.516128
precision	0.672996	0.516128
recall	0.366667	0.516128
mcc	0.453202	0.516128

Confusion matrix (at threshold=0.516128)

	Predicted as no	Predicted as yes
Labeled as no	6698	155
Labeled as yes	551	319

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

18.2 seconds

Metric details

	score	threshold
logloss	0.202325	nan
auc	0.926093	nan
f1	0.616172	0.337384
accuracy	0.913635	0.479097
precision	0.636977	0.479097
recall	1	0.00932094
mcc	0.565795	0.337384

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.202325	nan
auc	0.926093	nan
f1	0.585971	0.479097
accuracy	0.913635	0.479097
precision	0.636977	0.479097
recall	0.542529	0.479097
mcc	0.540265	0.479097

Confusion matrix (at threshold=0.479097)

	Predicted as no	Predicted as yes
Labeled as no	6584	269
Labeled as yes	398	472

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	4
4_Default_NeuralNetwork	1

Metric details

	score	threshold
logloss	0.177428	nan
auc	0.94601	nan
f1	0.646399	0.281485
accuracy	0.916872	0.506236
precision	0.670788	0.520433
recall	1	0.000236811
mcc	0.606017	0.281485

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.177428	nan
auc	0.94601	nan
f1	0.587404	0.506236
accuracy	0.916872	0.506236
precision	0.666181	0.506236
recall	0.525287	0.506236
mcc	0.546633	0.506236

Confusion matrix (at threshold=0.506236)

	Predicted as no	Predicted as yes
Labeled as no	6624	229
Labeled as yes	413	457

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

<< Go back